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Abstract 



^- ■ Logic programming is a powerful paradigm for programming autonomous agents in dynamic do- 

CNj | mains, as witnessed by languages such as Golog and Flux. In this work we present ALPprolog, an 

expressive, yet efficient, logic programming language for the online control of agents that have to 

reason about incomplete information and sensing actions. 
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1 Introduction 



Programming autonomous agents that behave intelligently is one of the key challenges 



of Artificial Intelligence. Because of its declarative nature, and high level of abstraction, 
logic programming is a natural choice for this task. This is witnessed by e.g. the two major 
exponents of agent programming languages that are based on classical logic programming, 
namely Golog (Leve sque et al. 1997[ ) and Flux dThielsch er 2005a). 

Both these languages combine a language for specifying the agent's behaviour with an 
axiomatic theory that describes the agent's environment. In the case of Golog the strat- 
egy language is procedural in nature (though implemented in Prolog), and the action 



theory is the classical Situation Calculus ([McCarthy and Hayes 1969 1 in Reiter's version 



dReiter 200 lal) . For Flux the strategy language is full classical logic programming, and the 
action theory is the more recent Fluent Calculus (Thielscher 19991 1. 

In a recent work (Dresch er et al. 20091 we have developed Agent Logic Programs (ALPs), 
a new declarative strategy language that is based upon a proof calculus in the style of clas- 
sical SLD-resolution. Contrary to Golog and Flux the ALP framework is parametric in 
the action theory: any background theory that allows to infer when an action is appli- 
cable, and what the effects of the action are, can be used. Exploiting this generality we 
have recently (Thielscher 2010b) been able to give a semantics for the BDI-style language 
AgentSpeak (Bordini et a l. 20071 ). Another distinctive feature of the theoretical framework 
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is the elegant handling of incomplete information for offline planning via disjunctive sub- 
stitutions. By default, ALPs are combined with our new Unifying Action Calculus (UAC) 
(lThielscher20111 l that encompasses the major logical action calculi, including both the 
Situation Calculus and the Fluent Calculus, as well as many planning domain description 
languages. The ALP formalism stays entirely within classical logic. 

The implementation of any fragment of the ALPprolog framework consists of (1) an 
implementation of the proof calculus, and (2) an action theory reasoner. Existing mature 
Prolog technology can be used out of the box for (1) unless disjunctive substitutions enter 
the picture. For (2) we can also exploit existing technology: E.g. Golog implements a 
fragment of the Situation Calculus, and Flux handles a fragment of the Fluent Calculus. 
In dDre scher et al. 2009) the implementation of a Description Logic -based fragment of the 
Fluent Calculus is described. 

In this work we present ALPprolog, where the underlying action theory is an essentially 
propositional version of the Fluent Calculus in the UAC that includes a simple, yet power- 
ful model of sensing. ALPprolog is intended for the online control of agents, where actions 
are immediately executed. This starkly contrasts with offline reasoning, where agents may 
make assumptions to see where these are leading. ALPprolog was developed specifically 
for the efficient handling of large ground state representations, something that we consider 
to be practically useful. To this end ALPprolog combines strong-points of Golog and Flux: 

• From Golog it takes the representation of the agent's state knowledge in full propo- 
sitional logic via prime implicates; and 

• From Flux it takes the principle of progression: The agent's state knowledge is up- 
dated upon the execution of an action. In standard Golog the agent's initial state 
knowledge is never updated|_J Instead, queries referring to later time-points are 
rewritten until they can be evaluated against the initial state knowledge, something 
which becomes a hindrance to good performance as the sequence of executed actions 
grows. 

We emphasise that ALPprolog is an agent programming language in the spirit of classi- 
cal logic programming in Prolog: The straightforward operational semantics provides the 
programmer with a powerful means to actively determine the sequence of actions that an 
agent executes. ALPprolog 2 1 can be obtained at [alpprolog . sourcef orge . net) 

The remainder of this paper is organised as follows: In Section [2] we recall the basics 
of the ALP framework, and in Section[3]we introduce ALPprolog. We evaluate the perfor- 
mance of ALPprolog in SectionHl and conclude in Section[5] 

2 ALPs in a Nutshell 

The purpose of agent logic programs is to provide high-level control programs for agents 
using a combination of declarative programming with reasoning about actions. The syn- 
tax of these programs is kept very simple: standard (definite) logic programs (see e.g. 
p.W. Lloyd 1987| l) are augmented with just two special predicates, one — written do ( a ) 

1 But there is a version of Golog where the initial state is periodically updated iSardina and Vasso s~2()05) . 

2 The name is a play on ALPs, propositional logic, and the implementation in plain Prolog. 
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— to denote the execution of an action by the agent, and one — written ? ( <p ) — to verify 
properties against (the agent's model of) the state of its environment. This model, and how 
it is affected by actions, is defined in a separate action theory. This allows for a clear sepa- 
ration between the agent's strategic behaviour (given by the agent logic program itself) and 
the underlying theory about the agent's actions and their effects. Prior to giving the formal 
definition, let us illustrate the idea by an example agent logic program. 

Example 1 

Consider an agent whose task is to find gold in a maze. For the sake of simplicity, the states 
of the environment shall be described by a single fluent (i.e., state property): At(u, x) to 
denote that u G {Agent, Gold} is at location x. The agent can perform the action Go(y) 
of going to location y , which is possible if y is adjacent to the current location of the 
agent. The following ALP describes a simple search strategy via a given list of locations 
(choice points) that the agent may visit, and an ordered collection of backtracking points. 
We follow the Prolog convention of writing variables with a leading uppercase letter. 

explore (Choicepoints, Backtrack) :- % finished, if 
? (at (agent, X) ) , ? (at (gold, X) ) . % gold is found 

explore (Choicepoints, Backtrack) : - 
? (at (agent, X) ) , 

select (Y, Choicepoints, NewChoicepoints) , % choose a direction 
do(go(Y)), % go in this direction 

explore (NewChoicepoints, [X | Backtrack] ) . % store the choice 

explore (Choicepoints, [X | Backtrack] ) :- % go back one step 
do (go (X) ) , 
explore (Choicepoints, Backtrack) . 

select (X, [X|Xs] ,Xs) . 

select (X, [Y|Xs] , [Y|Ys] ) :- select (X, Xs, Ys) . 

Suppose we are given a list of choice points C , then the query :- explore (C, []) lets 
the agent systematically search for gold from its current location: the first clause describes 
the base case where the agent is successful; the second clause lets the agent select a new 
location from the list of choice points and go to this location (the declarative semantics and 
proof theory for do (a) will require that the action is possible at the time of execution); 
and the third clause sends the agent back using the latest backtracking point. ■ 

The example illustrates two distinct features of ALPs: (1) The agent strategy is defined 
by a logic program that may use arbitrary function and predicate symbols in addition to the 
signature of the underlying action theory. (2) The update of the agent's belief according to 
the effects of its actions is not part of the strategy. Formally, ALPs are defined as follows. 

Definition 1 

Consider an action theory signature £ that includes the pre-defined sorts ACTION and 

FLUENT, along with a logic program signature II. 

• Terms are from E U II. 

• If p is an n-ary relation symbol from IT and ti, ...,t„ are terms, then p(ti,...,t„) 
is a program atom. 
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• do (a) is a program atom if a is an ACTION term in S. 

• ? (cp) is a program atom if 9? is a state property in £, that is, a formula (represented 
as a term) based on the FLUENTS in E. 

• Clauses, programs, and queries are then defined as usual for definite logic programs, 
with the restriction that the two special atoms cannot occur in the head of a clause. ■ 

2.1 Declarative Semantics: Program + Action Theory 

The semantics of an ALP is given in two steps. First, the program needs to be "tempo- 
ralised," making explicit the state change that is implicit in the use of the two special 
predicates, do ( a ) and ?(</?). Second, the resulting program is combined with an action 
theory as the basis for evaluating these two special predicates. The semantics is then the 
classical logical semantics of the expanded program together with the action theory. 

Time is incorporated into a program through macro-expansion: two arguments of sort 
TIMH 3 ! are added to every regular program atom p(x), and then p(x, si, S2) is under- 
stood as restricting the truth of the atom to the temporal interval between (and includ- 
ing) si and S2 ■ The two special atoms receive special treatment: ? (ip) is re-written to 
Holds(ip, s), with the intended meaning that ip is true at s; and do (a) is mapped onto 
Poss(a, 8%, S2), meaning that action a can be executed at si and that its execution ends 
in S2- The formal definition is as follows. 

Definition 2 

For a clause H : - Bi , . . . , B„ (n > 0), let si, . . . , s n +i be variables of sort TIME. 

• For i = 1, . . . , n, if Bj is of the form 

- p(ti, . . . ,t m ), expand to P(ti, . . . ,t m , Si,s i+1 ) . 

- do (a) , expand to Poss(a, Si, Si + i) . 

- ? ( (p ) , expand to Holds(ip, Si) A s»+i = Sj . 

• The head atom H = p (ti, . . . , t TO ) is expanded to P{t\, . . . ,t m ,sx, s n +i)- 

• The resulting clauses are understood as universally quantified implications. 
Queries are expanded exactly like clause bodies, except that 

• a special constant So — denoting the earliest time-point in the underlying action theory 
— takes the place of s± ; 

• the resulting conjunction is existentially quantified. ■ 



Example 1 (cont.) 

The example program of the preceding section is understood as the following axioms, 
which for notational convenience we have simplified in that all equations between TIME 



3 Which specific concept of time is being used depends on how the sort TIME is defined in the underlying action 
theory, which may be branching (as, e.g., in the Situation Calculus) or linear (as, e.g., in the Event Calculus). 
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variables have been applied and then omitted. 

(V)Explore(c, 6, si,si) C Holds(At(Agent, x), s) A Holds (At(Gold, a;), s\) 
(V)Explore(c, b, s\, S4) c Holds(At(Agent, a;), s\) A Select(y, c, c', si, s 2 )A 

Poss(Go(y), s 2 ,s 3 ) AExplore(c', [x\b], S3,s 4 ) 

(V)Explore(c, [x\b], s\, S3) C Poss(Go(a;), si, s 2 ) A Explore(c, 6, S2,s 3 ) 

(V)Select(a;, [a;|a;'],a;',si, si) C true 

(V)Select(a;, [y|a/], [y|y'], si, s 2 ) C Select(a;, x',y', 81,82) 

TTze resulting theory constitutes a purely logical axiomatisation of the agent's strategy, 
which provides the basis for logical entailment. For instance, macro-expanding the query 
: - explore (C, [ ] ) from the above example results in the temporalised logical for- 
mula (3s)Explore(C, [], So, s) . If this formula follows from the axioms above, then that 
means that the strategy can be successfully executed, starting at Sq , for the given list of 
choice points C . Whether this is actually the case of course depends on the additional 
action theory that is needed to evaluate the special atoms Holds and Poss in a macro- 
expanded program. ■ 

Macro-expansion provides the first part of the declarative semantics of an agent logic 
program; the second part is given by an action theory in form of a logical axiomatisation 
of actions and their effects. The overall declarative semantics of agent logic programs is 
given by the axiomatisation consisting of the action theory and the expanded program. 

Let us next introduce the fragment of the UAC corresponding to the Fluent Calculus. 
The UAC that is used to axiomatise the action theory is based on many-sorted first order 
logic with equality and the four sorts TIME, FLUENT, OBJECT, and ACTION. By convention 
variable symbols s, /, x, and a are used for terms of sort TIME, FLUENT, OBJECT, and 
ACTION, respectively. Fluents are reified, and the standard predicate Holds : FLUENT x 
TIME indicates whether a fluent is true at a particular time. The predicate Poss(a, s\, s 2 ) 
means that action a can be executed at si and that its execution ends in ,s 2 . The number 
of function symbols into sorts FLUENT and ACTION is finite. 

Definition 3 (Action Theory Formula Types) 

We stipulate that the following formula types are used by action theories: 

• State formulas express what is true at particular times: A state formula <D[s] in s 
is a first-order formula with free variables s where 

— for each occurrence of Holds(f, s) we have s e s ; 

— predicate Poss does not occur. 

A state formula is pure if it does not mention predicates other than Holds . 

• A state property <f> is an expression built from the standard logical connectives and 
terms F(x) of sort FLUENT. With a slight abuse of notation, by Holds(<p, s) we 
denote the state formula obtained from state property <j> by replacing every occur- 
rence of a fluent / by Holds(f, s) . In an expanded program n we always treat 
Holds((f>, s) as atomic. State properties are used by agent logic programs in ? ( Phi ) 
atoms. 

• The initial state axiom is a state formula <j>(So) in So , where Sq denotes the initial 
situation. 
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• An action precondition axiom is of the form 

(V)Poss(A(x),si,S2) = n A [si] A s 2 = Do(A(x),si), 

where 71^4 [si] is a state formula in si with free variables among s\,x . This axiom 
illustrates how different actions lead to different situation terms Do(A(x), s±) . Sit- 
uations constitute the sort TIME in the Fluent Calculus and provide a branching time 
structure. 

• Effect axioms are of the form 

Poss(A(x), si, s 2 ) D 

V(3jTfc)(**[«i] A (V/)[(\/ / = ,fe V (Holds(f, Sl ) A/\/ ^ fffci )) 

fe i j 

= /fo«s(/,s 2 )]). 

Such an effect axiom has fc different cases that can apply — these are identified 
by the case selection formulas $fe[si] which are state formulas in S\ with free 
variables among s\,x,yi . The fk% (and g^j , respectively) are fluent terms with 
variables among x, ijk and describe the positive (or, respectively, negative) effects 
of the action, given that case k applies. 

• Domain constraints are universally quantified state formulas (Vs)<5[s] in s. 

• Auxiliary axioms are domain-dependent, but time-independent, additional axioms 
such as e.g. an axiomatisation of finite domain constraints. 

An action theory T> is given by an initial state axiom T>\^ A , finite sets 2?p oss and £>Eff ec ts 
of precondition and effect axioms. Moreover domain constraints 2?d c and auxiliary axioms 
2? aux may be included. For illustration, the following is a background axiomatisation for 
our example scenario as a basic Fluent Calculus theory in the UAC. 

Example 1 (cont.) 
Our example program can be supported by the following domain theory. 

• Initial state axiom 

Holds(At(Agent, 1), S* ) A Holds (At (Gold, 4), 5 ) 

• Precondition axiom 

Poss(Go(y),si,s 2 ) = (3x) (Holds (At(Agent, x),s\) A (y = x + 1 V y = x - 1)) 
As 2 =Do(Go(j/),Si) 

• Effect axiom 

Poss(Go(j/),si,s 2 ) 3 

(3x) (Holds (At(Agent, x),Si) A 

[(V/)Holds(/, s 2 ) = (Holds(/, si) V / = At(Agent, y)) A / # At(Agent, x)]). 

Given this (admittedly very simple, for the sake of illustration) specification of the back- 
ground action theory, the axiomatisation of the agent's strategy from above entails, for 
example, (3s) Explore([2, 3, 4, 5], [], So, s) . This can be shown as follows. First, observe 
that the background theory entails 

Holds(At(Agent, 4), S) A Holds (At (Gold, 4), S), 
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where S denotes the situation term Do(Go(4), Do(Go(3), Do(Go(2), So))) ■ It follows 
that Explore([5], [3, 2, 1], S, S) according to the first clause of our example ALP. Con- 
sider, now, the situation S' = Do(Go(3), Do(Go(2), So)) , then action theory and strategy 
together imply 

Holds (At(Agent, 3), S') A Select(4, [4, 5], [5], S", S') A Poss(Go(4), S', S) 

By using this in turn, along with Explore([5], [3, 2, 1], S, S) from above, according to the 
second program clause we obtain Explore([4, 5], [2, 1], S", S) . Continuing this line of rea- 
soning, it can be shown that 

Explare([3,4,5],[l],Do(Go(2),S ),S) 
and hence, Explore([2, 3, 4, 5], [], So, S) 

This proves the claim that (Els) Explore([2, 3, 4, 5], [], So, s) . On the other hand e.g. the 
query (3s) Explore([2, 4], [], So, s) is not entailed under the given background theory: 
Without location 3 among the choice points, the strategy does not allow the agent to reach 
the only location that is known to house gold. ■ 



2.2 Operational Semantics: Proof Calculi 

We have developed two sound and complete proof calculi for ALPs that both assume the 
existence of a suitable reasoner for the underlying action theory (Dres cher et al. 2009I) . 

The first proof calculus is plain SLD-resolution, only that Holds - and Poss -atoms are 
evaluated against the action theory. This calculus is sound and complete if the underlying 
action theory has the witness property: That is, whenever V 1= (3x)4>(x) then there is a 
substitution 8 such that T) N (\/)<j>{x)6 . Note that in general action theories may violate 
the witness property, as they may include disjunctive or purely existential information; 
consider e.g. the case Holds(At(Gold, 4), So) V Holds(At(Gold, 5), So) , where the exact 
location of the gold is unknown. 

Hence the second proof calculus, intended for the general case, resorts to constraint 
logic programming, and the notion of a disjunctive substitution: Still assuming that the 
gold is located at one of two locations the query V \= (Bx)Holds(At(Gold, x) can now 
be answered positively via the disjunctive substitution x — > A\/ x — > 5 . Disjunctive 
substitution together with the respective principle of reasoning by cases are a powerful 
means for inferring conditional plans. 

For the online control of agents, however, assuming a particular case is unsafe. But if we 
use the plain SLD-resolution-based ALP proof calculus on top of action theories that lack 
the witness property we obtain a nice characterisation of cautious behaviour in a world of 
unknowns (albeit at the cost of sacrificing logical completeness). For ALPprolog this is the 
setting that we use. 

In both proof calculi we adopt the "leftmost" computation rule familiar from Prolog. 
This has many advantages: First, it simplifies the implementation, as this can be based 
on existing mature Prolog technology. Second, state properties can always be evaluated 
against a description of the "current" state. Last, but not least, this ensures that actions are 
executed in the order intended by the programmer — this is of no small importance for the 
online control of agents. 
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3 ALPprolog 

We next present ALPprolog — an implementation of the ALP framework atop of action 
theories in a version of the Fluent Calculus that 

• uses (a notational variant of) propositional logic for describing state properties; 

• is restricted to actions with ground deterministic effects; and 

• includes sensing actions. 

The intended application domain for ALPprolog is the online control of agents in dy- 
namic domains with incomplete information. 



3.1 ALPprolog Programs 

An ALPprolog program is an ALP that respects the following restrictions on the ? (Phi ) 
atoms in the program: 

• All occurrences of non-fluent expressions in <j> are positive. 

• So called sense fluents S(x) that represent the interface to a sensor may only occur 
in the form ? ( s ( X ) ) . Sense fluents are formally introduced below. 

Because ALPprolog programs are meant for online execution the programmer must en- 
sure that no backtracking over action executions occurs, by inserting cuts after all action 
occurrences. Observe that this applies to sensing actions, too. It is readily checked that — 
after the insertion of cuts — the ALP from example[T]satisfies all of the above conditions. 



3.2 Propositional Fluent Calculus 

In this section we introduce the announced propositional fragment of the Fluent Calculus. 
The discussion of sensing is deferred until section [53J 

For ease of modelling we admit finitely many ground terms for fluents and objects, 
instead of working directly with propositional letters. An action domain T> is then made 
propositional by including the respective domain closure axioms. For actions, objects, and 
fluents unique name axioms are included — hence we can avoid equality reasoning. 

The basic building block of both the propositional Fluent Calculus and ALPprolog are 
the so-called prime implicates of a state formula <f>(s) : 

Definition 4 (Prime Implicate) 

A clause i/> is a prime implicate of <f) iff it is entailed by <f>, is not a tautology, and is not 

entailed by another prime implicate. 

The prime implicates of a formula are free from redundancy — all tautologies and im- 
plied clauses have been deleted. For any state formula an equivalent prime state formula 
can be obtained by first transforming the state formula into a set of clauses, and by then 
closing this set under resolution, and the deletion of subsumed clauses and tautologies. 

Prime state formulas have the following nice property: Let be a prime state formula, 
and let i/> be some clause (not mentioning auxiliary predicates); then ip is entailed by <fi 
if and only if it is subsumed by some prime implicate in <f> , a fact that has already been 
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exploited for Golog (Reiter 2001a; IReiter 200 lbi This property will allow us to reduce 
reasoning about state knowledge in ALPprolog to simple list look-up operations. 
Formally the propositional version of the Fluent Calculus is defined as follows. 

Definition 5 (Propositional Fluent Calculus Domain) 

We stipulate that the following properties hold in propositional Fluent Calculus domains: 

• The initial state £>i n ; t is specified by a ground prime state formula. 

• The state formulas <j>(si) in action preconditions Poss(a, Si, 82) = <H s i) A s 2 = 
Do(a,si) are prime state formulas. 

• The effect axioms are of the form 

Poss(A(x),si, s 2 ) D 

\f($k[si] A (V/)[(\/ / = f ki V (Holds(f, *i) A /\ / # flfjy)) 

k i j 

= Holds(f,s 2 )}), 

where each $fc[si] is a prime state formula. This implies that existentially quanti- 
fied variables that may occur in case selection formulas (cf. definition [3]l have been 
eliminated by introducing additional cases. 



Only so-called modular domain constraints ( Herzig and Varzinczak 2007 1 may be 
included. Very roughly, domain constraints are modular if they can be compiled into 
the agent's initial state knowledge, and the effect axioms ensure that updated states 
also respect the domain constraints. In the Fluent Calculus this holds if the follow- 
ing two conditions are met (Thiels cher 201 11 1: Condition dTJ, says that for a state that 
is consistent with the domain constraints and in which an action A(x) is applica- 
ble, the condition $i[5] for at least one case i in the effect axiom for A holds. 
Condition (O requires that any possible update leads to a state that satisfies the do- 
main constraints. Formally, let S, T be constants of sort Time. £>d c the domain 
constraints, T>? oss the precondition axioms, and £>Effects the effect axioms. The fol- 
lowing must hold for every action A(x) : There exists i = 1, . . . , n such that 

\=V<k[S\Aw A [S\A(3i?i)$i[S\, (1) 

and for every such i , 

h Pdc [S] A ir A [S] A T 4 [S, T] D V dc [T] . (2) 

Non-modular, fully general domain constraints greatly complicate reasoning. 
Auxiliary time-independent axioms may be included if they can faithfully be repre- 
sented in the Prolog dialect underlying the implementation. This deliberately sloppy 
condition is intended to allow the programmer to use her favourite Prolog library. 
However, we stipulate that auxiliary predicates occur only positively outside of Z? aux 
in the action domain V in order to ensure that they can safely be evaluated by Prolog. 
They also must not occur in the initial state formula at all. The update mechanism 
underlying ALPprolog can handle only ground effects. Hence, if auxiliary atoms are 
used in action preconditions, case selection formulas of effect axioms, then it is the 
burden of the programmer to ensure that these predicates always evaluate to ground 
terms on those variables that also occur in the action's effects. 



10 Conrad Drescher and Michael Thielscher 

On the one hand clearly every propositional Fluent Calculus domain can be transformed 
to this form. On the other hand it is well known that in general compiling away the quan- 
tifiers in a state formula can result in an exponential blow-up, as can the conversion to 
conjunctive normal form. We believe that the simplicity of reasoning with prime impli- 
cates outweighs this drawback. 

Propositional action domains can still be non-deterministic. For example, for an appli- 
cable action two different cases may be applicable at the same time. The resulting state 
would then be determined only by the disjunction of the cases' effects. What is more, it 
would be logically unsound to consider only the effects of one of the cases. For the online 
control of agents in ALPprolog we stipulate that for an applicable action at most a single 
case applies, greatly simplifying the update of the agent's state knowledge. 

Definition 6 (Deterministic Propositional Fluent Calculus) 

A propositional Fluent Calculus domain is deterministic if the following holds: Let a be 
an applicable ground action. Then there is at most one case of the action that is applicable 
in the given state. 

For example, an action theory is deterministic if for each effect axiom all the cases are 
mutually exclusive. Next assume we have an applicable deterministic action with e.g. two 
case selection formulas <j>(s) and ^<fi(s) , where neither case is implied by the current 
state. Here, instead of updating the current state with the disjunction of the respective 
effects, ALPprolog will employ incomplete reasoning. 

3.3 Propositional Fluent Calculus with Sensing 

We make the following assumptions concerning sensing: At any one time, a sensor may 
only return a single value from a fixed set 1Z of ground terms, the sensing results. However, 
the meaning of such a sensing result may depend upon the concrete situation of the agent. 

Example 1 (cont.) 

Assume that now one of the cells in the maze contains a deadly threat to our gold-hunting 
agent. If the agent is next to a cell containing the threat she perceives a certain smell, 
otherwise she doesn 't: She can sense whether one of the neighbouring cells is unsafe; but 
the actual neighbouring cells are only determined by the agent's current location. ■ 

Definition 7 (Sensor Axiom) 

A sense fluent S(x) is a unary fluent that serves as interface to the sensor. We assume the 

sort SENSEFLUENT to be a subsort of sort FLUENT. A sensor axiom then is of the form 

(\/s,x,y)Holds(S(x),s) = \J x = R A <fi(x, y, s) A tf>(x, y, s), 

Ren 

for a ground set of sensing results 1Z . Here <j>(x, y, s) is a prime state formula that selects 
a meaning of the sensing result R , whereas the pure prime state formula ip(x, y, s) de- 
scribes the selected meaning. We stipulate that sensor axioms (which are a form of domain 
constraint) may only be included if they are modular. 

Clearly 4>(x,y,s) should be chosen so as to be uniquely determined in each state. If 
auxiliary axioms are used in (j>(x, y, s) then again the programmer must ensure that these 
evaluate to ground terms in order that a ground state representation can be maintained. 
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Example 1 (cont.) 
The following is the sensor axiom for our gold-hunter: 

(V)Holds(PerceiveSmell(a;), s) = 

x = true A Holds (At (Agent, y), s) A Neighbours (y, z) A \J Holds (Threat At (z), s) 

V 

x = false A Holds (At (Agent, y), s) A Neighbours(y, z)aA -Holds (ThreatAt(z), s)) 

z£z 



Theoretically, the combination of sensing with the online control of an agent is quite 
challenging: It is logically sound to to consider the disjunction of all possible sensing re- 
sults for offline reasoning. In the online setting, however, upon the observation of a sensing 
result we henceforth have to accept this result as being true; that is, at runtime we add the 
result to the action theory, something which is logically unsound. On the other hand, it also 
does not make sense to stipulate that the sensing result be known beforehand. 

3.4 Action Theory Representation 

We continue by describing how the underlying action theory is represented in ALPprolog. 
As basic building block we need a representation for prime state formulas. For notational 
convenience we will represent (-^)Holds(f, s) literals by the (possibly negated) fluent 
terms only, and, by an abuse of terminology, we will call such a term (->)/ a fluent literal. 
A convenient Prolog representation for such a state formula is a list, where each element is 
either a literal (i.e. a unit clause) or a list of at least two literals (a non-unit clause). In the 
following we call such a list a Pi-list. 

Definition 8 (Action Theory Representation) 

Action theories as defined in definition [6] are represented in ALPprolog as follows: 

• The initial state is specified by a Prolog fact initial_state (Pi-List) ., 
where P I -Li st mentions only ground fluent literals. Domain constraints other than 
sensor axioms have to be compiled into PI -List. 

• a Prolog fact action (A, Precond, EffAx) ., for each action a, has to be in- 
cluded, where 

— A is an action function symbol, possibly with object terms as arguments; 

— Precond is a Pi-list, the action's precondition; 

— EffAx is a list of cases for the action's effects with each case being a pair 
Cond-Ef f, where the effect's condition Cond is a Pi-list, and the effects 
Ef f are a list of fluent literals; and 

— all variables in EffAx also occur in Precond. 



• 



If present, auxiliary axioms 2? aux are represented by a set of Prolog clauses. The 
predicates defined in the auxiliary axioms must be declared explicitly by a fact 
aux (Aux) . , where Aux denotes the listing of the respective predicate symbols. 
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The sensor axioms are represented as Prolog facts sensor_axiom ( s (X) , Vals ) . , 
where 

• s is a sense fluent with object argument X; and 

• Vals is a list of Val-Index-Meaning triples, where 

— Val is a pair X-result_i, where result_i is the observed sensing re- 
sult; 

— Index is a Pi-list consisting of unit clauses; and 

— Meaning is a Pi-list, mentioning only fluent literals and only variables from 
Val and Index. 

The sense fluents have to be declared explicitly by a fact sensors (Sensors) ., 
where Sensors is a listing of the respective function symbols. This is necessary in order 
to distinguish sense fluents, ordinary fluents, and auxiliary predicates in Pi-lists. 

3.5 Reasoning for ALPprolog 

Reasoning in ALPprolog works as follows: For evaluating the program atoms we readily 
resort to Prolog. The reasoner for the action theory is based on the principle of progression. 
Setting out from the initial state, upon each successful evaluation of an action's precondi- 
tion against the current state description, we update the current state description by the 
action's effects. 

Reasoning about the action comes in the following forms: 

• Given a ground applicable action a, from the current state description <fi(si) and 
the action's positive and negative effects compute the description of the next state 
ij>(s2) (the update problem). 

• Given a description cf)(s) of the current state, check whether {4>(s)}UV aux N tp(s) , 
where ■ip(s) is some state formula in s , but not a sense fluent (the entailment prob- 
lem). 

• For a sensing action, i.e. a query Holds(S(x), s) , integrate the sensing results ob- 
served into the agent's state knowledge (the sensing problem). 

In the following we consider each of these reasoning problems in turn. 

3.5.1 The Update Problem 

It turns out that solving the update problem is very simple. Let State be a ground Pi- 
List, and let Update be a list of ground fluents. The representation of the next state is then 
computed in two steps: 

(1) First, all prime implicates in State that contain either an effect from Update, or 
its negation, are deleted, resulting in Statel. 

(2) The next state NextState is given by the union of Statel and Update. 

Starting from a ground initial state only ground states are computed. 

The correctness of this procedure can be seen e.g. as follows: In dLiu et al. 2 006 : Dres cher et al. 200"9b 
algorithms for computing updates in a Fluent Calculus based upon Description Logics have 
been developed. The above update algorithm constitutes a special case of these algorithms. 
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3.5.2 The Entailment Problem 

When evaluating a clause ijj against a ground prime state formula <f> , tp is first split into 
the fluent part ipi , and the non-fluent part ip2 ■ It then holds that %/} is entailed by <fi if 
there is a ground substitution 8 such that 

• ipiQ is subsumed by some prime implicate in <f> ; or 

• some auxiliary atom P(x)6 from ip2 can be derived from its defining Prolog 
clauses. 

Computing that the clause ipi is subsumed by <fi can be done as follows: 

• If -01 is a singleton, then it must be a prime implicate of <fi (modulo unification). 

• Otherwise there must be a prime implicate in <j> that contains ipi (modulo unifica- 
tion). 

Hence the entailment problem for ALPprolog can be solved by member, memberchk, 
and subset operations on sorted, duplicate-free lists. 

The following example illustrates how reasoning in ALPprolog can be reduced to sim- 
ple operation on lists. It also illustrates the limited form of reasoning about disjunctive 
information available in ALPprolog: 

Example 2 (Disjunctions and Substitutions in ALPprolog) 

Assume that the current state is given by [ [at (gold, 4 ) , at (gold, 5) ] ]. Then 
the query ? ( [at (gold, X) ] ) fails, because we don't consider disjunctive substitutions. 
However, on the same current state the query ? ( [ [at (gold, X) , at (gold, Y) ] ] ) 
succeeds with X=4 and Y=5. ■ 

3.5.3 The Sensing Problem 

Sensing results have to be included into the agent's state knowledge every time a sensing 
action is performed, i.e. a literal ? ( s (X) ) is evaluated. This works as follows: 

• First we identify the appropriate sensor axiom sensor_axiom ( s (X) , Vals ) . 

• Next we identify all the [X-result_i] -Index-Meaning triples in Vals such 
that result_i matches the observed sensing result, and unify X with result_i. 

• We then locate the unique Index-Meaning s.t. the current state entails Index. 

• Finally, we adjoin Meaning to the current state and transform this union to a Pi-list. 

3.6 Soundness of ALPprolog 

At the end of section [3~3l we have already mentioned that adding sensing results to the ac- 
tion theory at runtime makes the subsequent reasoning logically unsound wrt. the original 
program plus action theory. If we add the set of sensing results observed throughout a run 
of an ALPprolog program, however, then we can obtain the following soundness result: 

Proposition 1 (Soundness of ALPprolog) 

Let II be a ALPprolog program on top of an action domain T> . Let £ be the union of 
the sensor results observed during a successful derivation of the ALPprolog query T with 
computed answer substitution 6 . Then T> U II U E t= TO . 
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Proof (Sketch) 

It is well-known that SLD-resolution is sound for any ordinary program atom. A query 
(3)Holds(<fi, s), where <f> is not a sense fluent, is only evaluated successfully if there is 
a substitution 9 such that T> N (\/)Holds(<fi, s)9 . Assume we observe the sensing result 
Ri S 1Z for a sense fluent S(x, s) . In general we have (cf. Definition |7): 

V N {V)Holds(S(x, s)) A \f x = R, but V ¥ (V)Holds(S(x, s))Ax = R l . 

RGTZ 

For soundness, we have to add the observed sensing result as an additional assumption to 
the theory: V U {Holds(S{R t , a))} 1= (\/)Holds{S{x, s)) A x = R t . □ 



4 Evaluation 

We have evaluated the performance of ALPprolog via the so-called Wumpus World ( |Russell and Norvig 2003) 1 
that is a well-known challenge problem in the reasoning about action community. Essen- 
tially, the Wumpus World is an extended version of the gold-hunter domain from exam- 
ple[T] The main features that make it a good challenge problem are incomplete information 
in the form of disjunctions and unknown propositions, and reasoning about sensing results. 
We have used both Flux and ALPprolog to solve Wumpus Worlds of size up to 32 x32l 4 1 
We have done this using three different modellings: 

(1) In dThielscher 2005bt a Flux model is described that uses quantification over vari- 
ables — this is beyond ALPprolog. 

(2) We have evaluated both languages on a ground model. 

(3) We have artificially increased the size of the ground model by making the connec- 
tions between cells part of the state knowledge. 

A first observation is that both languages roughly scale equally well in all models. Using 
(1) Flux is slightly faster than ALPprolog using (2). Let us then point out that on ground 
models Flux and ALPprolog maintain the same state representation: Flux also computes 
the prime implicates. On the encoding (2) ALPprolog is roughly one order of magnitude 
faster than Flux, whereas on (3) the difference is already two orders of magnitude. The key 
to the good performance of ALPprolog then is that it handles large state representations 
well: By encoding states as sorted lists (of lists) some of the search effort necessary in 
Flux can be avoided. If, however, we use Flux' capability of handling quantified variables 
in the state knowledge for a more concise encoding, then ALPprolog and Flux are again 
on par, with Flux even having slightly the edge. In general, we expect ALPprolog to excel 
on problem domains that feature large state representations that are not easily compressed 
using quantification. 

It has already been established that Flux gains continuously over standard Golog the 
more actions have to be performed (Thielscher 2005a). As ALPprolog scales as well as 
Flux the same holds for ALPprolog and Golog. The version of Golog with periodically 
progressed state knowledge is slightly slower than Flux (Sardi na and Vas sos 2005). 



The distribution of ALPprolog contains the Wumpus World example for both ALPprolog and Flux. 



ALPprolog — A New Logic Programming Method for Dynamic Domains 1 5 

Let us also compare ALPprolog, Flux, and Golog from a knowledge representation per- 
spective: Both ALPprolog and Flux allow the programmer to define new auxiliary predi- 
cates for the agent strategy that are not present in the action theory, a practically very useful 
feature that is missing from Golog. Also, the propositional variables used in Golog instead 
of the finitely many ground terms used in ALPprolog make it hard for the programmer to 
fully exploit the power of Prolog's unification mechanism. In this regard Flux, on the other 
hand, excels in that the programmer can include fluents containing (possibly quantified) 
variables in the agent's state knowledge. Contrary to ALPprolog and Golog, however, Flux 
does not support arbitrary disjunctions. 

5 Conclusion and Future Work 

In this work we have presented ALPprolog, an efficient logic programming language for 
the online control of autonomous agents in domains that feature incomplete information 
and sensing. On the one hand, it can be argued that the state-of-the-art languages Golog 
and Flux already successfully address this application domain. On the other hand, we have 
shown that ALPprolog excels because of its efficient reasoning with large ground state 
representations, something that we expect to be quite useful in practice. 

For future work, there are two interesting directions: On the one hand it would be nice 
to extend ALPprolog to offline planning. The disjunctive substitutions in the general ALP 
proof calculus provide a powerful form of reasoning about conditional plans, or planning 
in the presence of sensing in the sense of (|Levesque 1996). 



On the other hand we plan to fruitfully apply ALPprolog in the domain of General 
Game Playing. General Game Playing (Geneser eth et al. 20 05) is a new exciting AI re- 
search challenge aiming at the integration of manifold AI techniques: A program (also 
called a player) is given an axiomatisation of the rules of a game. The player then computes 
a strategy/heuristic that it uses to play and hopefully win the game. The main challenge of 
General Game Playing consists of constructing suitable heuristics. 

However, at its base the player also needs a means to represent, and reason about, the 
state of the game. Up to now the games played in General Game Playing have been re- 
stricted to complete information dLove et al. 20081 ) — but clearly games with incomplete 
information constitute a bigger challenge (Thielscher 2010a). We intend to include tech- 
niques from ALPprolog into the successful Flux-based Fluxplaver(Schif fel and Thielscher 20071 ). 
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